WanKeeper: Efficient Coordination at WAN scale

Traditional coordination services for distributed applications do not scale well over wide-area networks (WAN): centralized coordination fails to scale with respect to the increasing distances in the WAN, and distributed coordination fails to scale with respect to the number of nodes involved. We argue that it is possible to achieve scalability over WAN using a hierarchical coordination architecture and a smart token migration mechanism, and lay down the foundation of a novel design for a flexible-consistent coordination framework, called WanKeeper. We implemented a prototype of WanKeeper based on the ZooKeeper API and deployed it over WAN as a proof of concept. Our evaluation in the context of BookKeeper and Shared Cloud-backed File System (SCFS) use cases shows that WanKeeper provides multiple folds improvement in write/update performance in WAN compared to ZooKeeper. Keywords-Coordination; Distributed systems; Wide-area networks; Scalability